Web Survey Bibliography
Title Using eye-tracking to understand how fourth grade students answer matrix items
Author Maitland, A.; Sun, H.; Caporaso, A.; Tourangeau, R.; Bertling, J.; Almonte, D.
Year 2015
Access date 23.08.2016
Full text PDF (522 KB)
Abstract
Eye-tracking has been used to better understand the survey response process. For instance, eye-tracking has been used to identify questions that are difficult to comprehend, how to present long lists of response options, and to measure the length of fixation on definitions in Web surveys. Matrix questions have been commonly used in Web surveys as well as in other types of surveys.The literature shows respondents took less time to answer questions when they were presented in a matrix than when they were presented individually across separate pages or screens. The use of matrix questions, however, may also be associated with several undesirable outcomes, including higher breakoff rates, higher missing data rates, and straightlining. Relatively little is known about how children answer matrix questions. This paper demonstrates how eye-tracking was used to determine the feasibility of using matrix questions to measure the background characteristics of students in the National Assessment of Educational Progress (NAEP) questionnaire. Fourth grade students were answered both matrix and discrete versions of questions on tablet computers while wearing real-world eye tracking glasses. This study addresses four research questions related to the use of eye-tracking to test survey questions. First, we examine whether matrix items require more effort to answer than discrete items for fourth grade students. Second, we investigate how the processing of sub items change within a matrix. Third, we examine how the processing of questions change over time. In order to address these research questions, we examine difference in the mean number of fixations per word and the mean duration per word for matrix and discrete questions. Overall, the study finds support for the use of matrix questions with fourth grade students in the NAEP. Implications for the use of eye-tracking equipment to evaluate survey questions are also discussed.
Access/Direct link FCSM Research Conference Homepage (Abstract) / (Full text)
Year of publication2015
Bibliographic typeConferences, workshops, tutorials, presentations
Web survey bibliography - Tourangeau, R. (45)
- Reducing speeding in web surveys by providing immediate feedback; 2017; Conrad, F.; Tourangeau, R.; Couper, M. P.; Zhang, C.
- Using eye-tracking to understand how fourth grade students answer matrix items; 2015; Maitland, A.; Sun, H.; Caporaso, A.; Tourangeau, R.; Bertling, J.; Almonte, D.
- Measurement Properties of Web Surveys; 2015; Tourangeau, R.
- Providing a Deadline for Response: Results from Two Recent Experiments; 2014; Kaiser, A., Walston, J. T., Medway, R., Ye, C., Tourangeau, R.
- The Relations Among Different Cognitive Shortcuts in Surveys; 2013; Tourangeau, R., Medway, R., Presser, S.
- Anonymity and Confidentiality; 2012; Tourangeau, R.
- Modes of Data Collection; 2012; Tourangeau, R.
- Database Lookup in Web Surveys; 2012; Couper, M. P., Zhang, C., Conrad, F. G., Tourangeau, R.
- Question Order Effect: A Web Survey Experiment with Paradata; 2012; Ye, C., Tourangeau, R.
- Should I Stay or Should I go: The Effects of Progress Feedback, Promised Task Duration, and Length of...; 2011; Yan, T., Conrad, F. G., Tourangeau, R., Couper, M. P.
- Nonresponse Error, Measurement Error, And Mode Of Data Collection: Tradeoffs in a Multi-mode Survey...; 2011; Sakshaug, J. W., Yan, T., Tourangeau, R.
- The psychology or survey response. An ASA webinar; 2010; Tourangeau, R.
- The psychology of survey response, 2nd Edition; 2010; Tourangeau, R., Bradburn, N. M.
- Professional Web Respondents and Data Quality; 2010; Conrad, F. G., Tourangeau, R., Couper, M. P., Zhang, C.
- Increasing Respondents' Use of Definitions in Web Surveys; 2010; Peytchev, A., Conrad, F. G., Couper, M. P., Tourangeau, R.
- Interactive Interventions in Web Surveys Can Increase Respondent Conscientiousness; 2009; Conrad, F. G., Tourangeau, R., Couper, M. P., Kennedy, C.
- Experimenting With Speech Interactive Voice Response, Touchtone Data Entry and the Web for the National...; 2009; Cantor, D., Brick, P. D., Tourangeau, R., Conrad, F. G.
- Interactive feedback can improve accuracy of responses in web surveys; 2009; Conrad, F. G., Couper, M. P., Tourangeau, R., Galesic, M.
- Response Order and Response Distributions: The Format of the Response Options in a Web Survey; 2009; Tourangeau, R., Conrad, F. G., Couper, M. P., Balter, O.
- Improving the Design of Complex Matrix Questions; 2009; Couper, M. P., Tourangeau, R., Conrad, F. G.
- Survey Methodology (Wiley Series in Survey Methodology); 2009; Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., Tourangeau, R.
- Social desirability bias in CATI, IVR and Web surveys: The effects of mode and question sensitivity; 2008; Kreuter, F., Presser, S., Tourangeau, R.
- Experiments Exploring the Relationship (or Lack Thereof) Between Nonresponse Error and Measurement Error...; 2008; Tourangeau, R., Groves, R. M., Presser, S., Toppe, C., Kennedy, K., Yan, T.,
- The Impact of the Spacing of the Scale Options in a Web Survey; 2008; Kennedy, C., Tourangeau, R., Conrad, F. G., Couper, M. P., Redline, C. D.
- Fast times and easy questions: the effects of age, experience and question complexity on web survey...; 2008; Yan, T., Tourangeau, R.
- Sensitive questions in surveys; 2007; Tourangeau, R., Yan, T.
- What is Sexual Harassment? It Depends on Who Asks! Framing Effects on Survey Responses; 2007; Galesic, M., Tourangeau, R.
- The Impact of the Visible: The Design of Web Surveys; 2007; Tourangeau, R., Couper, M. P., Conrad, F. G.
- How the shape and format of input fields affect answer; 2007; Couper, M. P., Tourangeau, R., Conrad, A.
- Experiments in Producing Nonresponse Bias ; 2006; Groves, R. M., Couper, M. P., Presser, S., Singer, E., Tourangeau, R., Piani Acosta, G., Nelson, Li.
- Color, Labels, and Interpretive Heuristics for Response Scales; 2006; Tourangeau, R., Couper, M. P., Conrad, F. G.
- An Experimental Comparison Of Web And Telephone Surveys; 2005; Fricker, S., Galesic, M., Tourangeau, R., Yan, T.
- Visual Context Effects in Web Surveys; 2005; Couper, M. P., Conrad, F. G., Tourangeau, R.
- Does Voice Matter? An Interactive Voice Response (IVR) Experiment; 2004; Couper, M. P., Singer, E., Tourangeau, R.
- Spacing, Position, and Order: Interpretive Heuristics for Visual Features of Survey Questions; 2004; Tourangeau, R., Conrad, F. G., Couper, M. P.
- Picture This! Exploring Visual Effects in Web Surveys; 2004; Couper, M. P., Tourangeau, R., Kenyon, K.
- Web-based data collection; 2003; Tourangeau, R.
- Cognitive Aspects of Survey Measurement and Mismeasurement; 2003; Tourangeau, R.
- Understanding the effects of audio-CASI on self-reports of sensitive behavior; 2003; Couper, M. P., Singer, E., Tourangeau, R.
- Humanizing self-administered surveys: experiments on social presence in web and IVR surveys; 2003; Tourangeau, R., Couper, M. P., Steiger, D. M., de Rouvray, C.
- Use and non-use of clarification features in web surveys; 2003; Tourangeau, R., P., Couper, M. P., Conrad, F. G., Baker, R. P.
- What They See Is What We Get: Response Options for Web Surveys; 2003; Couper, M. P., Tourangeau, R., Conrad, F. G., Crawford, S. D.
- Self-administered questions by telephone: Evaluating interactive voice response; 2002; Tourangeau, R., Steiger, D. M.,
- Collecting sensitive information with different modes of data collection; 1998; Couper, M. P., Tourangeau, R., Smith, T. W.
- Cognitive science and survey methods; 1984; Tourangeau, R.